Using ergodic theory to assess the performance of ecosystem models.

Ecosystem simulation models are designed to assess the flux of energy, water, carbon and nitrogen according to a given vegetation type. The reliability of the modeled results is determined by model validations. Model validations are typically done using classical statistical methods like regression analysis of predicted versus observed values, paired t statistics and error assessment procedures to characterize the quality of current and future model predictions. All these validation efforts concentrate on static aspects of the model but fail to describe the model dynamics. In this paper, we introduce methods from ergodic theory to analyze the dynamic behavior of ecosystem models. We describe (1) how the attractor representation of model behavior can be reconstructed from a time series of model outputs, and (2) what we can learn from the attractor to assess the model dynamics. As an application example, we provide simulation results for two important pine forest ecosystems in Austria, i.e., 23 Cembran pine and 16 Scots pine stands. These stands were simulated with three model parameterizations: one representing a generic, evergreen needle-leaf forest, and two species- specific parameter sets, one for Cembran pine and one for Scots pine. First, we applied standard validation methods to get static measures of model accuracy and precision. Next, we used ergodic theory to assess model dynamics. A comparison of both analyses reveals important issues related to model dynamics, such as the finding that the occurrence of instabilities in model behavior may not be detected by standard validation methods. Using ergodic theory, we were able to reconstruct the attractors of model behavior. In the attractor describing model dynamics for Cembran pine, simulated with the generic, evergreen needle-leaf forest parameter set, we detected instability in model behavior. We identified this instability as a riddled basin configuration, which is a strong indicator for the occurrence of a chaotic model behavior that may result in random predictions. Our results suggest that ergodic theory is a useful tool for assessing inconsistencies in the dynamics of ecosystem model simulations that have not been detected by standard statistical validation methods.